Plots of Prediction Accuracy Based on Test Data Set

The Model used is the Gradient boosting model trainined on basic features from with the orange data mining app.

The test data set represents 10% of the data. The test data hasnt been seen by the trainined model.

What we are looking for are clusters of inaccuracy that imply the model. What we want to see is incorrect predictions that are not grouped in an area of space or concentrated in a particular sheep that imply a weakness in the model.

data <- get_predictions("individual_model_predictions.tab")
plot_all_wavefront(data,"S9","Test data")
plot_all_wavefront(data,"S12","Test data")
plot_all_wavefront(data,"S15","Test data")
plot_all_wavefront(data,"S17","Test data")
plot_all_wavefront(data,"S18","Test data")
plot_all_wavefront(data,"S20","Test data")

Plots of Prediction Accuracy Based on Test AND Training Data - i.e. All Data

While not an accurate way to assess the model performance, by evaluating the model on data it has seen before, the 3d plot should make more sense in what is looks like.

data <- get_predictions("predictions_3d_all_data.tab")
plot_all_wavefront(data,"S9", "All data")
plot_all_wavefront(data,"S12","All data")
plot_all_wavefront(data,"S15","All data")
plot_all_wavefront(data,"S17","All data")
plot_all_wavefront(data,"S18","All data")
plot_all_wavefront(data,"S20","All data")

Some conclusions

Some cursory conclusions can be drawn when Comparing the All Data predictions to the 3D.html that show the ground truth.

  1. S9 inaccuracies driven by a large proportion of epi-only labels the basic model has difficulty predicting on.

  2. S12 has good prediction accuracy in areas where there is a large “At Least Endo” label. Inaccuracies are evenly spread otherwise (which is good)

  3. S18. Inaccuracies group together where there are “At Least Intra” and “Epi Only” labels.

  4. S20 (control sheep) has high level of predictor accuracy as it has majority of “No Scar” label. Where there are inaccuracies it is distributed in space and not clustered around a small amout of “At least Endo” labels (which is good)

Overall there is no bias in a particular sheep or area in space. Inaccuracies primarily driven by the basic model and its ineffectiveness in outer layers.